Network Regions: Alternatives to the Winner-Take-All Structure

Winner-take-all (WTA) structures are currently used in massively parallel (connectionist) networks to represent competitive behavior among sets of alternative hypotheses. However, this form of competition might be too rigid and not be appropriate for certain applications. For example, applications that involve noisy and erroneous inputs might mislead W T A structures into selecting a wrong outcome. In addition, for networks that continuously process input data, the outcome must dynamically change wi th changing inputs; W T A structures might " lockin " on a previous outcome. This paper offers an alternative competition model for these applications. The model is based upon a meta-network representation scheme called network regions that are analogous to net spaces in partitioned semantic networks. Network regions can be used in many ways to clarify the representational structure in massively parallel networks. This paper focuses on how they are used to provide a flexible and adaptive competition model. Regions can be considered as representational units that represent the conceptual abstraction of a collection of nodes (or hypotheses). Through this higher-level abstraction, regions can better influence the collective behavior of nodes wi th in the region. Several AI applications were used to test and evaluate this model. I . I N T R O D U C T I O N Winner-take-all (WTA) structures [ l ] represent competitive behavior among sets of alternative hypotheses in massively parallel networks [2, 3, 4). These networks consist of large numbers of simple processing elements that give rise to emergent collective properties. The behavior of such networks has been shown to closely match human cognition in many tasks, such as natural language understanding and parsing, learning, speech perception and recognition, speech generation, physical skil l modeling, vision and others. In a W T A structure, whenever there is any activat ion, the structure forces only the node (which may represent a hypothesis) wi th the highest output level to remain activated, while the other nodes die out. This mechanism allows one to define "decision points" that Figure 1. WTA structure that represents competing weak fricatives. In the example, the l ink weights for inhibit ion and activation links are arbitrari ly set to be —0.6 and 0.33 respectively and the decay factors are set to 0.3. Figure 1 is the final state after the inputs —voiced, 4labial , and f f r i ca t i ve were activated and the network was relaxed. As the figure shows, the W T A structure enabled /(/ to compete and suppress the other candidates. 380 KNOWLEDGE REPRESENTATION output suppress nodes with lower output. There are two parameters that influence this competitive behavior — the inhibition link weights and the nodes' decay factor. The link weights define the degree of competition. Lower link weights represent a milder, slower form of competition. Higher link weights represent stronger, quicker competition. The decay factor, on the other hand, controls how well a network retains the decision made by the WTA structure and allows the network to reset itself in the absence of inputs. In this paper, these two parameters are assumed to be uniform.